Digital computer systems generally consist of a short-term memory system, a long-term memory system, a processor, and a set of input/output (I/O) devices for interacting with the outside world. The short-term memory system is generally a volatile memory such as a random access memory that provides for fast access to needed data while the computer system is being used. The long-term memory system is generally a larger and slower non-volatile memory system such as a hard disk drive that can store information while the computer system is not being used.
A computer system operates by having a processor read in a sequence of computer instructions from the short-term memory system and then sequentially executes that sequence of computer instructions. Conditional branch instructions within the computer instruction stream will determine any changes in the sequence of instructions that will be executed. Results from the processed computer instructions may be stored in the short-term memory for temporary storage. Processing results may also be stored in the long-term non-volatile memory system for permanent storage. Under the control of the computer instructions, the processor will access the input/output devices to interact with the humans and the outside world. There are limitless varieties of input/output devices including keyboards, video display systems, printers, networks, cursor control devices, audio generation system, etc.
When considering the performance of a digital computer system, many people tend to focus on the execution speed of the processor in the computer system. However, a high-speed computer processor without a matching high-speed memory system that supplies instructions and data just as fast as the processor executes the instructions will eventually become starved of the needed instructions and data. Thus, a high-performance computer system design requires both a high-speed processor for quickly executing instructions and a high-speed memory system to supply instructions and data to the high-speed processor.
One of the most popular techniques for providing high-speed memory services is to create a small high-speed cache memory system that is tightly integrated with (and sometimes created on the same die as) the computer processor. Typically, a high-speed cache memory system duplicates an area of a much larger but slower main memory system. Provided that the needed instructions or data are currently represented within the small high-speed cache memory system, the processor will be able to execute at full speed (or close to full speed since sometimes the cache runs slower than the processor, but much faster than the slow memory). When a cache ‘miss’ occurs (the required instruction or data is not available in the high-speed cache memory), then the computer processor must wait until the slower main memory system responds with the needed instruction or data.
Cache memory systems provide an effective means of creating a high-speed memory subsystem that can support high-speed processors. Conventional cache memory system may be implemented within a network device to improve performance. However, conventional high-speed cache memory systems typically require large amounts expensive low density memory technologies that consume larger amounts of power than standard dynamic random access memory (DRAM). For example, static random access memory (SRAM) technologies are often used to implement high-speed cache memory systems. Static random access memory (SRAM) integrated circuits typically cost significantly more than commodity DRAM, consume much more power than DRAM, and are much less dense (less bits per integrated circuit die area) than DRAM.
The most significant drawback of conventional cache memory systems in the context of network devices is that conventional cache memory systems do not guarantee high-speed access to desired information. Specifically, a conventional cache memory system will only provide a very fast response when the desired information is represented in the high-speed cache memory. If the desired information is not represented in the cache memory system, then the processor will need to access the main memory system.
With a good cache memory system design that incorporates clever heuristics that ensure that the desired information is very likely to be represented in the cache memory, a memory system that employs a high-speed cache will provide a very good response time on average. However, if the desired information is not represented in the cache memory, then a slow fetch to the main (slower) memory system will be required. In networking applications that require a guaranteed response time, a conventional cache memory system will not provide a satisfactory high-speed memory solution. Thus, other means of improving memory system performance must be employed.
One simple method that will provide a guaranteed response time is to construct the entire memory system from high-speed SRAM. However, this method is very expensive, occupies a large amount of board space, generates significant amounts of heat, and draws excessive amounts of power.
Due to the lack of a guaranteed performance from conventional cache memory systems and the very high cost of building entire memory systems from high-speed SRAM, it would be desirable to find other ways of creating high-speed memory systems. In particular, it would be desirable to create high-speed memory systems for network devices and other applications that require guaranteed performance. Ideally, such a high-speed memory system would not require large amounts of SRAM since SRAM is expensive, low-density, consumes a relatively large amount of power, and generates significant amounts of heat.